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ABSTRACT 



Governments and other stakeholders are increasingly interested in assessing the skills of their 
adult populations in order to monitor how well prepared they are for the challenges of the modern 
knowledge-based society. The current paper provides an overview of the conceptual framework guiding 
the development of the assessment of literacy in the OECD’s Programme for the International Assessment 
of Adult Competencies (PIAAC). It builds upon earlier conceptions of literacy from the International 
Adult Literacy Survey (IALS) in the 1990s and the Adult Literacy and Life Skills Survey (ALL) in 2003 
and 2006 to facilitate an appropriate assessment of the broad range of literacy skills required for the 21 st 
century. The framework broadens the definition of literacy to make it relevant to the information age, in 
particular, by including the skills of reading in digital environments. 



RESUME 



Les pouvoirs publics et autres parties prenantes s’interessent de plus en plus a revaluation des 
competences de la population adulte dans un objectif de suivi de son etat de preparation face aux defis de 
la societe moderne de la connaissance. Le present article fournit une vue d’ensemble du cadre devaluation 
de la comprehension des textes du Programme pour revaluation internationale des competences des 
adultes (PIAAC) de l’OCDE, afin d’ identifier les taches de lecture et ecriture auxquelles les indi vidus sont 
confrontes dans le nouveau monde de l’information. Ce cadre s’appuie sur des definitions anterieures de la 
comprehension des textes, notamment celles de l’enquete International Adult Literacy Survey (IALS) 
menee dans les annees 1990 et de l’enquete Adult Literacy and Life Skills Survey (ALL) conduite en 2003 
et 2006 dans le but de permettre P evaluation adequate d’une vaste gamme de competences requises au 
21 eme siecle. Le cadre elargit la definition de la comprehension des textes en faisant place a de nouvelles 
formes de textes et en developpant une comprehension plus profonde des connaissances cognitives qui 
sous-tendent la comprehension des textes des adultes et du role que joue l’engagement dans des activites 
de lecture. 
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PIAAC LITERACY- CONCEPTUAL FRAMEWORK 



INTRODUCTION 



1 . With national economies facing growing unemployment as the new century ends its first decade, 
the issues of human capital development rise in importance. In a series of studies in the 1990s, the OECD, 
Statistics Canada, and Educational Testing Service demonstrated the importance of literacy skills for the 
effective functioning of labor markets and for the economic success and social advancement of both 
individuals and societies. Data from three rounds of the International Adult Literacy Survey (OECD & 
Statistics Canada, 2000) and the Adult Literacy and Life Skills Survey (OECD & Statistics Canada, 2005) 
revealed that low skills are found in all countries and that those low skills pose problems for individuals 
trying to cope with work and life in modern societies. 

2. The demands on literacy skills have always been increasing. As economies became increasingly 
industrialised, nations needed ever more educated populations and literacy skill took on greater 
importance. As nations progressed, the quantity and type of materials continued to expand and individuals 
found they were expected to use information from written materials in new and more complex ways. 

3. Now, as information-based economies are succeeding industrial-based economies, literacy is 
again being transformed. A new form of text, digital text, makes increased demands on readers, and 
changes the ways in which text is used. The amount of information available and its uncensored nature 
emphasise the abilities needed to connect, evaluate and interpret information. In addition, computer 
technologies have added new dimensions related to the nonlinear, recursive, and interactive nature of these 
environments. Because individuals now often move through the material in their own ways when searching 
for information, they very often create their own “texts” in the sense that the total set of information that 
each individual encounters is unique. Collectively, the skills required to effectively use digital information 
are less well understood than traditional print skills but suggest that we will need to expand our definition 
of what it means to be literate. 

4. A broadened view of literacy that includes skills and knowledge related to information and 
communication technologies is increasingly seen as an essential component of the knowledge, skills and 
attitudes that facilitate the creation of personal, social and economic well-being. (Kellner, 2002; 
Partnership for 21 st Century Skills, 2003; Rainie & Horrigan, 2005; Senn-Breivik, 2005). In the global 
economy, individuals and nations with information and communication technology skills will most likely 
prosper while those lacking them will struggle to compete. It is therefore essential that we understand more 
about the ways in which we use information and communication technologies and the associated outcomes 
of those uses. According to Murnane and Levy (1996), these “new basic skills” are needed by everyone 
regardless of their aspirations, regardless of whether they are male or female, and regardless of their social 
and economic backgrounds. 
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5. At the same time that we see these increasing demands on individuals, research, especially in the 
United States and Canada, has shown had many individuals still have difficulty with the underlying skills, 
such as vocabulary and fluency, that are the building blocks necessary to developing the higher levels of 
literacy. To have a full picture of literacy in any society, it is necessary to have more information about 
these individuals because these groups are at the greatest risk of experiencing negative social, economic 
and labor market outcomes. 

6. The Programme for the International Assessment of Adult Competencies is designed to assess the 
state of the skills of individuals and nations in this new information world. Clearly, one of those skills is 
literacy and the framework set out in this paper by the Literacy Expert Group for PIAAC reflects its 
understanding of what the literacy skills that are necessary in the 21 st century are. 

7. It is helpful to understand the process by which this document was developed. The Group met for 
three days in late May 2008 and reviewed the literacy framework that had guided the IALS and ALL 
assessments as well other reports and documents. General agreement was reached on expanding and 
reworking those frameworks to reflect new literacy demands. Stan Jones, as Chair of the Expert Group, 
prepared a draft which was then circulated to all members for comment. Mr. Jones then reworked the draft 
based on these comments to produce a version that was discussed at the June 2008 NPM meeting. After 
that NPM further drafts were developed and circulated for comments. The Literacy Expert Group met 
again in Valencia, Spain, in November, 2008 and reviewed a further draft. This final version was prepared 
following further discussion at an Expert Group meeting in March, 2009. 

8. In its discussion of component reading skills, the Expert Group determined that developing a 
detailed framework and set of measures around these skills was also important. While these measures are 
an important part of the PIAAC Literacy Framework, their nature is somewhat different than the measures 
of literacy-in-use developed here. The role of the components in an overall assessment of literacy is 
included in this framework, but the complete discussion of the measures and their justification and 
development can be found in a separate document approved by the Literacy Expert Group. 

9. The Group would like to express its considerable appreciation to Mary Lou Lennon for carefully 
and thoughtfully reviewing an early draft of the document and to Juliette Mendelovits and her item 
development team for their feedback. Our thanks also go to Jean-Pierre Jeantheau who served as a 
consultant at the first meeting and in the review. 

10. Members of the Literacy Expert Group: 

• Egil Gabrielsen - Norway 

• Jan Hagston - Australia 

• Stan Jones (Chair) - Canada 

• Pirjo Linnakyla - Finland 

• Hakima Megherbi - France 

• John Sabatini - USA 

• Monika Troster - Germany 

• Eduardo Vidal-Abarca - Spain 



6 




EDU/WKP(2009)13 



DEFINING LITERACY FOR PIAAC 



11. In any large-scale assessment, the definition of the domain to be measured provides a foundation 
for the design of the assessment and sets the boundaries for what will be included. Such a definition needs 
to be situated in the wider context of adult literacy, but constrained by the goals of the PIAAC project. 
When considering the construct definition for PIAAC, the Literacy Expert Group began by looking at 
previous assessments. Literacy assessments have been part of two major international surveys of adult 
skills: the International Adult Literacy Survey (LALS) in the 1990s and the Adult Literacy and Life Skills 
Survey (ALL) in 2003 and 2006. These were, in turn, informed by several prior national surveys. However, 
during the 20 years that these assessments cover, adults have faced new literacy opportunities and 
demands, especially as the use of -mail and other digital media has grown. Lor PIAAC to appropriately 
assess adult literacy in the 21 st century, it is necessary to broaden the construct to include new modes of 
text. PIAAC also provides an opportunity to deepen our understanding of the cognitive skills that underlie 
adult literacy and of the role that engagement plays in literacy. The Group noted that the PISA 2009 
definition specifically references both electronic texts and engagement (OECD: Programme in 
International Student Assessment). 

12. Given the opportunity to rethink literacy as a construct, the PIAAC Literacy Expert Group was 
guided by four principles: 

a) The definition must be descriptive and not normative. Many definitions of literacy set out an ideal 
state, or a goal for literacy action. The underlying construct for PIAAC has to recognise that 
individuals vary in their ability to succeed across the full range of literacy tasks they face in 
modern society. 

b) The definition should support an expanded conception of literacy, in these ways: 

• There should be an assessment of the underlying skills of those at the lower levels of 
literacy; 

• The range of texts to be considered should be broader than in previous assessments; in 
particular, the definition should include those texts often identified as electronic texts; and 

• The type literacy behaviors measured should go beyond simply using text for an 
immediate purpose, so as to enable a deeper understanding of literacy ability. 

c) At the same time, the definition should support a link to the LALS and ALL assessments to 
support the analysis of trends. 

d) Cognitive assessment alone is not sufficient to understanding the status of literacy in a particular 
society. It is also important to consider the engagement of individuals in literate activities. 

13. The Literacy Expert Group considered a number of definitions that had been recently proposed, 
and found the following definition from a UNESCO expert group in 2003 to be useful: 

Literacy is the ability to identify, understand, interpret, create, communicate and compute, using 
printed and written materials associated with varying contexts. Literacy involves a continuum of 
learning in enabling individuals to achieve their goals, to develop their knowledge and potential, 
and to participate fully in their community and wider society. 
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14. Some elements of this definition were beyond the scope of the current PIAAC assessment 
(though, perhaps, not future ones), especially the productive elements of creating and communicating. 
Computing was not appropriate for inclusion in the PIAAC literacy definition as it is in the domain of the 
Numeracy Expert Group. Other parts needed to be rephrased to better support an assessment. 

15. In the end, the Literacy Expert Group found that an expanded and re-ordered version of the 
IALS/ALL 1 definition would meet both the descriptive, expansive, and linking criteria it wanted for 
PIAAC. 

Literacy is understanding, evaluating, using and engaging with written texts to participate in 
society, to achieve one’s goals, and to develop one’s knowledge and potential. 

16. In the rest of this section, we set out what we understand by the key elements of this definition 
Written text 

17. Previous literacy assessments have focused primarily on informative texts of both continuous and 
non-continuous form. It is the intention of the new construct to expand the range of texts to include a 
greater variety of text types, such as narrative and interactive texts, and a greater variety of media. Until 
recently, most adult reading was of material printed on paper. Now, adults need to access and use text that 
is displayed on a screen of some kind, whether of a computer, a PDA, an ATM, or a Blackberry or iPhone. 
The PIAAC definition encompasses all these. 

18. It is worth noting that including electronic text opens the assessment to new types of text and 
content. While one can find examples of similar texts in paper, they are much less common in that form. 
Some of these novel form/content combinations include interactive texts, such as exchanges in comments 
sections of blogs or in e-mail response threads, multiple texts, whether displayed at the same time on a 
screen or linked through hypertext, and expandable texts, where a summary can be linked to more detailed 
information if the user chooses. 

Understanding 

19. A basic task for the reader is constructing meaning, large and small, literal and implicit, from 
text. This can be as basic as understanding the meaning of the words, to as complex as comprehending the 
underlying theme of a lengthy argument or narrative. Certainly, evaluating or using a text implies some 
level of understanding and so provides an indirect measure of it, but it is the intent of the PIAAC 
assessment to include some more direct measure of it. The components framework provides a construct to 
support basic understanding, but the literacy assessment, itself, should also include tasks that explicitly tap 
more complex understanding, such as the relation(s) between different parts of the text, the gist of the text 
as a whole, and insight into the author’s intent. Readers also have to understand the social function of each 
text and the way this influences structure and content. 

Evaluating 

20. Readers continually make judgments about a text they are approaching. They need to assess 
whether the text is appropriate for the task at hand, determining whether it will provide the information 
they need. They have to make j udgments about the truthfulness and reliability of the content. They need to 



1 The IALS / ALL definition reads: “Literacy is defined as a particular capacity and mode of behaviour: the ability to 
understand and employ printed information in daily activities, at home, at work, and in the community - to achieve 
one’s goals and to develop knowledge and potential.” 
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account for any biases they find in the text. And, for some texts, they must make judgments about the 
quality of the text, both as a craft object and as a tool for acquiring information. 

21. Because the provenance of many digital texts is obscure and because it is much easier to 
distribute digital texts widely and anonymously, such judgments are especially important for electronic 
texts. Sources for online information are more varied, ranging from authoritative sources to postings with 
unknown or uncertain authenticity. All information must be evaluated in terms of accuracy, reliability and 
timeliness, but this is particularly important with online material. 

Using 

22. Much adult reading is directed toward applying the information and ideas in a text to an 
immediate task or goal or to reinforce or change beliefs. Nearly all the tasks in previous international 
assessments have been of this kind. In some cases, using a text in this way requires just minimal 
understanding, getting the meaning of the words with some elementary recognition of structure (many 
menus, for example). In others, it requires using both syntactic and more complex structural understanding 
to extract the information. In all cases though, the reader approaches the text with a specific task in mind. 

Engaging with 

23. Many adults appear to read text only when some task requires them to do so. Others (sometimes) 
also read for the pleasure it brings them and for general interest. Some adults read only what others - 
employers, governments - make necessary, while others read things of their own choosing as well. That is, 
adults differ in how engaged they are with text and how much a role reading plays in their lives. Studies 
have found that engagement with (attitude toward and practice of) reading is an important correlate with 
the direct cognitive measures. As such it is necessary to understand these differences to get a full picture of 
adult literacy. 

Participate in society 

24. While earlier definitions referred to the role of literacy in “functioning” in society, the PIAAC 
use of “participating” is meant to focus on a more active role for the individual. Adults use text as a way to 
engage with their social surroundings, to learn about and to actively contribute to life in their community, 
close to home and more broadly. And for many adults, literacy is essential to then - participation in the labor 
force. In this, we recognise the social aspect of literacy, seeing it as part of the interactions between and 
among individuals. 

Achieve one ’s goals 

25. Adults have a range of needs they must address, from basic survival to personal satisfaction, to 
professional and career development, to participation in society. Literacy is increasingly complicit in 
meeting those needs, whether simply finding one’s way through shopping, or negotiating complex 
bureaucracies, whose rules are commonly available only in written texts. It is also important in meeting 
adult needs for sociability, for entertainment and leisure, for developing one’s community and for work 

Develop one ’s potential 

26. Surveys suggest that many adults engage in some kind of learning throughout their life, much of 
it self-directed and informal. Much of this learning requires some use of text and as individuals want to 
improve their life, whether at work or outside, they need to understand, use, and engage with printed and 
electronic materials. 
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The literacy domain 

27. While this definition gives us a broad picture of what PIAAC literacy proposes to measure, and 
what it does not, a more detailed discussion of the domain is needed to guide item development, to provide 
a basis for assessing the validity of the results, and to support analysis and reporting of the findings. In this 
we are able to build on the work from IALS and ALL, which has demonstrated the power of a few 
variables to account for much of the variance in individual scores. We do not propose to substitute a 
different type of model here, but to extend the old one to different types of text and new aspects of reading 
and to recast some elements to support a deeper understanding of adult reading. 

28. One aspect of the literacy construct, engagement, is best assessed through background questions 
and not in the cognitive assessment. As such, it is covered in a separate section that follows. As well, basic 
understanding is assessed in a set of measures known as the components test; it, too, is discussed 
separately. 

29. In creating items for an assessment the writer has two primary elements to manage. One is the 
text itself, with several important features that need to be considered. One of these features is the medium, 
format and type of text - its character as an object and another is the social setting in which the text is most 
naturally situated. The second primary element is the task the item writer sets for the assessment. A 
number of characteristics are known to play a role in the difficulty of these tasks and the writer needs to 
keep these in mind in constructing the items. These characteristics also play an important role in the 
analysis of the results and in descriptions of the different levels of literacy measured by the assessment. 
While we discuss these two elements separately, there is a close connection between them; in many ways a 
text determines the types of tasks that be asked using it and the item writer needs to balance the need for 
adequate coverage of text characteristics with the need for adequate coverage of task characteristics. 
Nonetheless, we discuss these two primary elements separately in this framework. 

Texts 

30. Any assessment of literacy requires texts to serve as the stimulus for the tasks that form the test. 
We have found it useful to organise the texts for PIAAC in a number of ways: 

a) Medium (print and digital) 

b) Lormat (continuous and non-continuous) 

c) Type (rhetorical stance) 

d) Physical layout (type of matrix organisation) 

e) Leatures unique to digital texts, and 

f) Social context. 

Medium 

31. A major development of PIAAC over previous adult surveys is the inclusion of digital (or 
electronic) texts. We discuss below some key features of digital texts that distinguish them in practice from 
printed texts. We recognise that many texts that would previously be encountered in printed form are now 
just as likely to be accessed digitally. Lor example, consumer information about a product that was once 
printed in a brochure may now be available only in some electronic format. Indeed, many of the 
respondents in PIAAC will encounter all the texts electronically, some of which will be simple copies of 
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printed texts. We want to distinguish digital texts not simply by the medium in which they occur, but by 
the use they make use text navigation and display features that found only through digital devices. Any text 
that could appear on a printed page exactly as it appears on a screen will be considered a print text; any 
text that could not appear on a printed page with all its features intact will be considered a digital text 2 . 

Format 

32. In IALS and ALL, and in PISA, texts were classified as continuous (prose) or non-continuous 
(document). This is an important distinction, as each format requires different text knowledge and a 
different approach to text processing. At the same time many actual texts involve some elements that are 
continuous and some that are non-continuous. Thus, the distinction is better made on the basis of what 
type(s) of text a task requires. 

a) Continuous. This type of text is conventionally made up of sentences formed into paragraphs. 
Some continuous texts include typographic features, such as indenting and headings, that signal 
the organisation of the text, but many do not. Examples of continuous texts include newspaper 
and magazine articles, brochures, manuals, e-mails and many web pages. 

b) Non-continuous. This type of text uses explicit typographic features, rather than paragraphs, to 
organise information. While there may be full sentences in some non-continuous texts, most 
consist of words or phrases organised by some kind of matrix arrangement. Tables, graphs, charts 
and forms are all examples of non-continuous texts. 

c) Mixed. This type of text has both continuous and non-continuous elements. Examples of mixed 
texts include web pages with a list of links, newspaper articles that incorporate line graphs or pie 
charts, and brochures with attached order forms. 3 

d) Multiple. Multiple texts consist of texts that have been generated and which make sense 
independent of each other. The texts are juxtaposed or loosely linked for a particular purpose. The 
relationships among the component texts need not be obvious. The texts may be contradictory or 
complementary. Such texts are common in digital settings, but are also found in print 
environments. 

Text type (Rhetorical Stance of the Text) 

33. The IALS/ ALL framework classified continuous texts by their rhetorical stance, since all share 
much the same structure (sentence and paragraph, with or without headings). But non-continuous texts also 
share the same rhetorical stances. Therefore, we propose to identify the stance of all types of text using the 
six categories employed in the IALS / ALL assessments. We would note that although narrative texts have 
been included as a text type in the past, few narrative texts have actually appeared in previous assessments. 
Our intent is that such texts should form a part of the PIAAC assessment. We have eliminated one text type 
that had been listed for IALS / ALL: Hypertext, as it is not a rhetorical category, but a structural type 
which will be included under electronic text for PIAAC. 



2 Although many pdf files are simply scanned copies of printed texts, the Acrobat reader application adds navigation, 
search and annotation features not available for the printed version. Thus, a pdf text would be a digital text by this 
description. 

3 There are texts, such as comics and graphic novels, which rely on graphic elements to carry important parts of the 
information. It seems to us that these are a special case of continuous text where the pictorial displays function to 
organise the sequence of ideas somewhat as the ordering of paragraphs does. 



11 




EDU/WKP(2009 ) 1 3 



34. The point of having rhetorical stance as a variable is not that there is any evidence that difficulty 
is affected by the stance, but as a way ensuring that a variety of texts are included on the assessment. 

35. The six types 4 of rhetorical stance for PIAAC are as follows: 

a) Description is the type of text where the information refers to properties of objects in space. A 
page of a manual that identifies the parts of some device, such as a Cuisinart, is a description, as is 
a verbal depiction of a piece of art. 

b) Narration is the type of text where the information refers to properties of objects in time. Stories 
recounted to make a point, such as fables, are narrations, as are texts about the steps what an 
individual took solve a problem. 

c) Exposition is the type of text in which the information is presented as composite concepts or 
mental constructs, or those elements into which concepts or mental constructs can be analyzed. 
The text provides an explanation of how the component elements interrelate in a meaningful 
whole. A text that explains the nature of some health problem or one that talks about the effect of 
climate change would be an exposition. 

d) Argumentation is the type of text that presents propositions as to the relationship among concepts 
or other propositions... An important subclassification of argument texts is persuasive texts. 
Newspaper editorials are one example, and advertisements are another. 

e) Instruction (sometimes called injunction) is the type of text that provides directions on what to do. 
Most equipment manuals contain instruction texts, as do other guides, such as those about first-aid 
or some leisure activity. 

f) Records are texts that are designed to standardise, present and conserve information without 
embedding in other stances. A table of standings in a sports league is an example of a record, as is 
a graph of the changes in oil prices. The minutes of a meeting constitute another type of record. 

36. The existing items from IALS and ALL that have been selected as linking items for PIAAC cover 
these categories well, with the exception of narrative texts as noted previously. A goal for the new PIAAC 
literacy items is to include a range of texts that represents all six of these types. 

Classifying non-continuous texts 

37. In additional to their rhetoric, non-continuous texts differ in their structural organisation. While 
all continuous texts have the same sentence-in-paragraph form, non-continuous texts differ in form and a 
major challenge to understanding their content is to understand the meaning of their forms. The IALS / 
ALL framework identified five types of non-continuous structures that we propose to adopt for PIAAC: 

a) Matrix Documents . This set of non-continuous text consists of four types of increasingly complex 
documents that have simple lists as their basic unit. 

• A simple list consists of a label and two or more items, where the label serves as the 
organizing category and the items all share at least one feature with the other items in the 
list. A basic shopping or ‘to do’ list is an example of this basic structure. 



4 These are taken almost directly from the IALS / ALL frameworks, as we saw no reason to change them. 
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• Next are combined lists , which consist of two or more simple lists. One list in a combined 
list is always primary and, as such, is ordered to facilitate looking up information within 
the list and then locating parallel information within the other lists. An email inbox is an 
example of the combined list structure with its related lists of sender names, subjects, 
dates and file sizes. 

• Intersected lists are the third type of matrix document and are composed of exactly three 
lists. Two of the lists form a row and column defining the cells which contain the third or 
intersected list. Television listings are a common example of an intersecting list with the 
channels and times defining the program content listed in each cell. 

• The fourth and most complex type of matrix document is the nested list. In order to 
economise on space, as well as to display comparative information, designers sometimes 
combine two or more intersecting lists to form a nested list. In a nested list, one type of 
information will be repeated in each of the intersecting lists. The intersecting list of 
unemployment rates, for example, may have separate entries under each month for males 
and females; in this case, gender would be nested under month. 

b) Graphic Documents. A major function of graphic documents is to provide a succinct visual 
summary of quantitative information. Included in this group of texts are pie charts, bar charts, and 
line graphs. While these appear to be very different types of documents on the surface, they all 
derive from, or can be transformed into, either a combined, intersecting, or nested list. 

c) Locative Documents. Like graphic documents, locative documents or maps portray information 
visually. Unlike graphic documents that display quantitative information, maps either portray the 
location of persons, places, or things in space, or depict characteristics of different geographic 
regions (e.g., types of vegetation or characteristics of a population). 

d) Entry Documents. In matrix and graphic documents, the author provides the information that must 
be read and used. In contrast, entry documents, or forms, require the reader to provide information 
that can range from very simple to complex. For example, the reader may be asked to simply 
check a box; write a single word, number, or phrase; or construct a series of phrases or sentences. 
Generally speaking, forms provide the reader with a label or category for which the reader is 
asked to provide specifics. 

e) Combination Documents. Some displays, especially graphic documents, rely on the use of other 
types of text for their interpretation. Maps and graphs, for instance, often include legends that 
display important information that must be read and understood. In addition, designers sometimes 
include more than one document for display or comparative purposes. 

Digital text features 

38. The new definition introduces digital texts as a type to be included in the assessment. It is 
important to note that digital texts use many of the same organizing principles and rhetoric as continuous 
and non-continuous texts. However, these texts have properties that function as navigation tools and 
features and distinguish them from paper and ink texts 5 . 



5 We believe there are analogs in standard printed texts to both of these, but they are much more common in digital 
text and are easier to construct and use in their digital form. We should also note that the use of computer-assisted text 
layout has resulted in paper-and-ink texts that are more complex than previously. One only needs to compare a 
newspaper from the 1960s to one from today to see the impact of this change. 
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a) Hypertext. Electronic texts may provide direct links to other texts and it may be necessary to 
follow these links to gain a full understanding of a topic. The physical process is normally to click 
with a mouse on a highlighted word or phrase in a text to access another text with (additional) 
ideas and information relevant to the highlighted part of the passage. We would initially 
distinguish two main types: 

• Index-like. The initial text is a list of topics, from which the reader selects one or more for 
additional information. A common example is a news site on the web which lists 
headlines on which the reader clicks to view the full stories. An e-mail inbox is another 
example of an index hypertext; clicking on an entry takes the reader to the full e-mail. In 
many workplaces such index hypertexts are used to retrieve information for a specific 
task. The entry screen lists categories of information from which the reader selects and 
the appropriate information is then retrieved and displayed by the computer (or other 
device; PDAs are often used in this way). Schedules and electronic calendars are another 
example of an index-like hypertext as the user can usually click on an entry in a schedule 
for additional information about that entry. 

• Text-embedded. In this type of hypertext a link is embedded in a complete text and the 
reader is taken to a second text that expands on the immediate topic. A common example 
is Wikipedia, in which an entry includes many links to other entries or to other web sites. 
A second kind of text-embedded hypertext includes links to other parts of the same text. 
For example, footnotes in electronic texts can often be accessed by clicking on the 
footnote number in the text. 

b) Interactive. In electronic text a reader often comes across texts that have been created by a series 
of authors. E-mail exchanges, where the sequence of messages is retained when replies are simply 
added to previous messages, are a common example of this interactive text. Comments sections of 
blogs or other web documents such as news sites that allow comments on stories are another 
instance. In these texts, later entries often cannot be understood without understanding prior 
contributions. 

c) Other Navigation Features. Digital documents typically have navigation features that differ from 
printed texts. While it seems intuitive to experienced readers to turn a page in a multi-page text, it 
is something that must be learned. In the same way, understanding that the scroll bar takes the 
reader to additional text, or that a digital text might require the reader to click on a next page (or 
previous page) button to move to new text are features that the digital reader must learn. 

Social contexts 

39. Adult reading normally is part of a social setting. Both the motivation to read and the 
interpretation of the content may be influenced by the context. As a result, a fair assessment must include 
material from a broad range of settings, so as to include some material that would familiar to any 
participant. 

40. The following content areas have proven useful in previous assessments: 

a) Work and occupation includes materials that deal in general with various occupations but not job- 
specific texts, finding employment, finance, and being on the job. 

b) Personal uses 
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• Home and family includes materials dealing with interpersonal relationships, personal 
finance, housing, and insurance. 

• Health and safety includes materials dealing with drugs and alcohol, disease prevention 
and treatment, safety and accident prevention, first aid, emergencies, and staying healthy. 

• Consumer economics includes materials dealing with credit and banking, savings, and 
advertising, making purchases, and maintaining personal possessions. 

• Leisure and recreation includes materials involving travel, recreational activities, and 
restaurants, as well as material read for leisure and recreation itself. 

c) Community and citizenship includes materials dealing with community resources and staying 
informed. 

d) Education and training includes materials that deal with opportunities for further learning. 

Tasks 

41. While any reading event is a complex cognitive operation, typically one aspect dominates any 
particular instance. A reader may be looking for a particular piece of information, but other aspects do 
come into play. The reader must judge whether the information source is reliable and whether it has the 
appropriate content. The reader must understand the semantic content of the text. But these are secondary 
to the main task of locating a specific piece of information. At other times, the reader may be primarily 
interested in understanding some phenomenon. Again judgments will have to be made and specific 
information will have to be considered, but the primary goal is broad understanding. The item writer 
attempts to capture this primary purpose in creating a task and any assessment must have tasks that focus 
on a variety of aspects of reading. 

Aspects of tasks 

42. Both IALS and ALL identified three broad aspects of tasks that readers were asked to carry out: 
those that require identification of pieces of information in the text, those that require connecting different 
parts of the text, and those that require some understanding of the text as a whole. 

Access and identify information in the text. On some occasions adults are simply seeking specific 
items of information from a text. What time does the movie start? How many cups of flour are 
used for this cake? What does this candidate propose to do about the roads? 6 Sometimes finding 
the needed information is relatively simple, as it is directly and plainly stated in the text. 
However, identify tasks are not necessarily easy ones. Lor some tasks, inferences may be required 
and rhetorical understanding may have to be called upon. Lor example, using a text to find the 
reasons for a change of action by the local government may require an understanding of how 
reasons are presented in text. In addition, sometimes more than one piece of information is 
required. IALS / ALL called these access and identify tasks locating (when only one piece of 
information was required) and cycling (when more than one is required). 



Note that this is a different task than answering should I vote for this candidate, but may be a step toward making 
that decision. 
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Most tasks that require only identification would be classified as “using” tasks. But some 
evaluation tasks might require identifying several distinct pieces of information which might be 
compared for their relevance in a particular situation. 

a) Integrate and interpret (relate parts of text to each other). Often tasks require the reader to 
understand the relation (s) between different parts of a text. These relations include problem- 
solution, cause-effect, category-example, equivalency, compare-contrast, and understanding 
whole-part relationships. To complete such tasks, the reader has to determine what the appropriate 
connection is. This may be explicitly signaled, as when the text states "the cause of X is Y”, or 
may require an inference by the reader. The parts to be related may be near each other in the text 
or may be in different paragraphs or even in different texts. In IALS / ALL such tasks were 
usually called integrate tasks. 

Compare and contrast, two basic evaluation steps, are examples of relating parts of a text to each 
other. Establishing what is the basis of a relationship between parts is a form of understanding at 
the sub-text level. Some parts of a text must be understood in the context of the rest of the text, as 
they take on essential elements of meaning from the larger text in which they occur. 

Readers are sometimes called upon to come to some understanding of a text as a whole. Lor 
example, the reader may need to determine the purpose of a text or comprehend its main theme. 
Again the text itself may make this explicit, as with a title or an introductory sentence or 
paragraph, but often it is something readers must discover on their own and produce a paraphrase 
or summary. 

b) Evaluate and reflect. Evaluation and reflection involve drawing on knowledge, ideas or values 
external to the text. The reader has to approach the text from outside, assessing the relevance, 
credibility, truthfulness of the information or argumentation presented in the text. The reader may 
also evaluate the purposefulness, register, structure or reader awareness of the text, or how 
successfully the writer is using the evidence and language to argue or persuade a reader. 
Evaluation is, particularly, important in reading electronic texts where the reader needs to be more 
alert to the text’s accuracy, reliability and timeliness. 

Readers also need to be aware of how the author of a text is attempting to persuade them to a 
particular end. Such meta-textual awareness is part of evaluating and reflecting on a text. And a 
reader needs to determine whether they are the intended audience for the text. 

Cognitive representations 

43. In understanding a text, a reader has to create a mental representation from linguistic materials of 
the text. For continuous texts these materials are words, phrases, and sentences. For non-continuous texts, 
the materials also include the matrix and list relations underlying the arrangement of text elements. There 
is a substantial body of research 7 on how readers create these representations for continuous texts, but, 
despite the ubiquity of non-continuous texts in adult reading, there is little work on how such 
representations are created for them 8 . 



7 In particular, see Garrod & Sanford, 1994; Gernsbacher & Foertsch, 1999; Oakhill & Garnham, 1988; Kintsch, 
1998; and Perfetti, 1994. 

8 There has been more interest in non-continuous texts among researchers concerned with digital texts. See, 
particularly, Rouet (2006), which is also useful on the processing differences of digital and print texts. 
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Factors that affect the construction of represen tations 

a) Transparency of the information. An important factor in task difficulty is the transparency of the 
information in the text. When the question refers explicitly to the superficial information (literal 
information), it’s easier to process. For some tasks, the needed information is explicitly signaled; 
a telephone number always has a particular form and may also be preceded by “Tel” in the text. 
The text may have a title, or the problem and solution may be directly labeled as such (explicit 
signal). 

h) Degree of complexity in making inferences 

Paraphrase. Readers have to process linguistic information by mobilizing their lexical and 
syntactic-semantic knowledge. Simple examples would be a task requiring readers to find 
information about the cost of an automobile in a table using the word ‘car’, or such as knowing 
that “ate” in a text indicates food. 

High level text inference. In a problem-solution text, for example, neither the problem nor the 
solution need to be explicitly signaled; rather the reader may have to infer what the problem 
(and/or solution) is from the text itself. And the reader cannot necessarily assume that the problem 
statement will precede the solution. 

Extra-textual inference. Some tasks require the reader to bring information from outside the text 
or from another text in order to understand parts of the text in question. For example, in a notice 
about local road repair projects, the reader may be expected to bring external knowledge about the 
types of roads in that area to understand the actual repair proposals. 

• Semantic complexity and syntactic complexity. Studies of both oral and written text have 
shown that the more concrete the information is the easier the task is. Tasks requiring the 
reader to identify persons, things or places tend to be easier than those that involve 
abstract properties, such as goals, conditions and purposes. The grammar structure of the 
sentence (question and text) could be more or less complex. For example, negative 
phrases are more complex than affirmative phrases. The presence of subordinate clauses 
(question or text) improves the complexity of syntactic processing. 

• Amount of information needed. The more information the reader needs from the text to 
complete the task, the more difficult that task will be. The distinction between locate and 
cycle tasks in IAFS / AFF was based on this (cycle tasks required the reader to make 
several independent identifications). As well, the amount of text that must be processed 
also plays a role in the difficulty of any task. 

• Prominence of the information. If the information the reader needs is located in a 
prominent location in the text (in the first or last sentence of a paragraph, in a main rather 
than subordinate clause, at the top or bottom of a list) it will be easier to access. 

• Competing information. The more potentially relevant information that the reader has to 
sift through to access the needed information, the more difficult the task will be. This is 
especially true if there is information in the text that might plausibly be appropriate, but is 
incorrect. For an obvious example, if a text has a telephone, a fax number and a mobile 
number, it will be more difficult to find the fax number, than if that were the only number 
in the text. 
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• Text features. The degree to which the reader has to construct relations among parts of the 
text affects difficulty. When there are large numbers of anaphoric references the reader 
must sort out and when text cohesion signals are absent, the reader will find the task more 
difficult. 

Consequences for test development 

44. The expanded definition of literacy has some important consequences for test development for 
PIAAC literacy. In the assessment structure proposed for PIAAC, there will be 48 literacy items on the 
computer based test and 24 on the paper on pencil test. Of these 29 of the computer and 19 of the paper and 
pencil items will be linking items drawn from the LALS / ALL pool. Only 19 computer and 5 paper and 
pencil items will be new. As the linking items test just part of the PIAAC framework, namely just the using 
written text portion, almost all item development needs to be directed to new parts of the framework. 

45. In order to have the items necessary for the final test a larger pool of items will need to be 
developed. As Table 1 shows, a total of 42 items from IALS and ALL have been selected for development 
as linking items. All 42 of these items will be authored for computer-based delivery and a subset of 25 
items will also be used for the paper and pencil version of the assessment. A total of 70 new items will be 
developed, 55 for the computer and 15 for paper and pencil. The 15 paper-and-pencil items may be subset 
of the 55 developed for the computer-based version of the assessment or they may be unique to the paper 
and pencil version. 

46. The items prepared in the development phase will be reviewed by countries and the consortium. 
Lorty-two linking items and up to 40 new items will be selected for the field test. Required numbers of 
items for the field test and main assessment are shown. 

Table 1 : Item development targets 



Development Field-test Main assessment 




Linking 

Items 


New 


Linking 

items 


New 


Linking 

items 


New 


Paper & pencil 


25 


15 


25 


10 


19 


5 


Computer-based 


42 


55 


42 


30 


29 


19 



Item development by task characteristics 

47. The Literacy Expert Group has specified overall targets in terms of the distribution of items 
across the three defined task characteristics: text type, context, and process. These distributions are shown 
in Tables 2 through 4 below. 



Table 2: Distribution of items by medium 



Medium 


Distribution 
of items (%) 


Print-based texts 

As used in previous studies: newspapers, magazine, books, brochures, manuals, 
announcements, letters, advertisements, etc. 

New to PIAAC: texts which combine prose and document elements, texts + images, and 
diagrams. 


60-70 

10-15 


Digital 

Including hypertext, interactive environments such as message boards and chat rooms, texts 
which combine prose and document elements, and texts + images 


20-25 



Note: Each category includes continuous, non-continuous and combined texts. 
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48. As Table 2 shows, the goal will be for electronic texts to comprise between 20 and 25 percent of 
the assessment. The bulk of the texts will be more traditional paper-based texts; however, within that 
category the expert group would like to include sets of tasks based on materials where prose and document 
or text and images are integrally related and respondents must use and/or integrate information from both 
sources. 

49. The distribution by context area shown in Table 3 is included to ensure that materials for the 
assessment represent a broad range of settings. As the motivation to read and interpretation of content may 
be influenced by context, including a broad range is meant to help ensure that no group of respondents will 
be either advantaged or disadvantaged based on their familiarity with, or interest in, a particular context. 

Table 3: Distribution by items by contexts 



Contexts 


Distribution of 
items 

(%) 


Work/occupation 


15 


Personal 

• Home and family 

• Health and safety 

• Consumer economics 

• Leisure and recreation 


40 


Community and citizenship 


30 


Education and training 


15 



50. Table 4 shows the targeted distribution of items by task aspects. The operations shown apply to 
each of the components of the definition, except for engagement. That is, a reader can access and identify 
information, integrate and interpret, and evaluate and reflect in order to understand, to evaluate and to use 
text. 



Table 4: Distribution by task aspects 



Task aspects 


Distribution of 
items 
(%) 


Access and identify information in the text 

• Locate 

• Cycle 


30-40 


Integrate and interpret (relate parts of text to each 
other) 

• Cause/effect 

• Compare/contrast 


40-50 


Evaluate and reflect 


15 



51. Although PIAAC does not plan to provide separate scales for continuous (prose) and non- 
continuous (document) tasks, it is important that there be approximately an equal number of tasks of each 
kind, recognizing that the framework does call for some tasks that use both continuous and non-continuous 
sources. These latter items, called mixed items here, are an important new feature of the PIAAC 
framework and the assessment must include items of this type. 
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Engagement 

52. The concept of reading engagement is an important one in adult literacy, referring to the degree 
of importance of reading to an individual and to the extent that reading plays a role in their daily life. 
Empirical studies with children and adults have shown that differences in engagement are systematically 
related to differences in performance on assessments. Engagement theory typically identifies 5 integrated 
aspects of the concept. 

a) Amount and variety of reading. The more one reads and the more different types of reading 
(purposes, types of text) one uses, the greater one is engaged with reading. 

b) Interest in reading. The more one seeks out reading as a means of obtaining information and for 
enjoyment, the greater one is engaged with reading. 

c) Control. The more one feels in control of what one reads and is able to direct one’s own reading, 
the greater the engagement. 

d) Efficacy. The more an individual feels able to read well, especially the confidence to read 
successfully new texts, the greater one is engaged with reading. 

e) Social interaction. The more one is interested in sharing reading experiences and seeks out others 
to talk about reading, the more one is engaged with reading. 

53. Experience in IALS and ALL has demonstrated that aspects 1 and 2, and to some extent 6, can be 
reliably evaluated through self-reports, but that adults do not, indeed may not be able to, reliably self-report 
control and efficacy. Questions appropriate for assessing engagement have been recommended for 
inclusion in the background questionnaire. 

Component assessment 

54. In previous surveys, the information on the reading abilities of adults with poor skills was often 
insufficient to get a proper understanding of their difficulties. Because they could answer so few of the 
items on the test, there was little on which to build a description of their abilities. The literacy framework 
for PLAAC includes a component test intended to provide that information. 

55. The Components Assessment is more fully described in a separate document. Because we regard 
it as a supplement to the main literacy assessment a summary of that framework is included here. 

56. The components framework builds upon a basic principle of learning to read, now widely 
researched and accepted internationally. That is, the comprehension or ‘meaning construction’ processes of 
reading are built upon a foundation of component skills and knowledge of how one’s writing system 
works. The evidence of this knowledge and skills can be captured in tasks that examine a reader’s ability 
and efficiency in processing the elements of the written language - letters/characters, words (and non- 
words), sentences, and larger, continuous text segments. 

57. A second principle guiding the components design is that the main interest is in whether the 
adults surveyed can apply their existing language and comprehension skills to the processing of printed 
texts. The components tasks are not designed to separately assess the level of language skills in the target 
print literacy writing system. It is assumed that the adults surveyed will have basic oral vocabulary, 
syntactic/ grammatical, and listening comprehension skills in the target language. We provide a component 
measure of basic oral vocabulary as an indicator that individuals surveyed have a threshold level of 
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language proficiency. However, independent measurement of language proficiency is not a basic feature of 
the component framework. 

58. A third principle of this model of reading is that the level of proficiency, efficiency, and 
integration of component skills is indicative of level and learning potential in reading development. As 
skills and knowledge accumulate, the ease of processing familiar text-based information increases. 
Component efficiency is typically indexed by assessing speed or rate of processing, as well as accuracy. As 
learners, we spend extra time, effort, and energy to solve problems that are novel. On familiar tasks, we 
can often respond, accurately, quickly, with seemingly little conscious effort. When the tasks are easy, we 
can spend more effort solving and learning from more complex problems and tasks. 

59. Work with this model has typically identified five components: 

• alphanumeric perceptual knowledge and familiarity, 

• word recognition, 

• word knowledge (vocabulary), 

• sentence processing, and 

• passage fluency. 

60. In skilled reading, these components are integrated to support literacy performance. During 
acquisition, even with adults, they may be measured separately, with different profiles having implications 
for learning, instruction, and policy. 

61. Two of these components, a) and b), are particular to the writing system of a language. Because 
some languages have more transparent writing systems than others, it is difficult to develop measures that 
could be used to compare different national populations. 9 As a consequence, the first two are offered as 
optional elements of the assessment. 

62. The components are seen as an integral part of the overall literacy assessment. They are 
necessary, though not sufficient elements of skilled reading. In analysis, the component scores should be 
reflected on the literacy distribution, as in Figure 1, to show the relation between component acquisition 
and literacy ability. 



9 For example, a component study with adults in Canada found significant differences in the transparency of the 
French and English writing systems which made comparisons of results in the two languages difficult. 
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Figure 1 : Proposed approach to reflecting components scores on the PIAAC literacy scale 




63. Details of each of these five components are available in the full Components Assessment 
framework. 

Developing the components assessment 

64. A prototype components assessment will be developed in English by the Consortium. Some parts 
may require nothing more than translation to be usable in other languages. Other elements may need to be 
developed specifically for some or all languages. For these, the English will serve as guidelines. The intent 
is to provide comparable assessments that will support international comparisons. 

Analysis 

65. It is anticipated that the scores on the PIAAC literacy assessment will be determined as they were 
for IALS and ALL and that the PIAAC scores will be equated to those on the previous surveys, using a 0- 
500 scale. Levels, similar to those used in IALS / ALL should also be computed and reporting should make 
use of these levels, of the 0-500 scale and of the components analysis. 

66. Descriptions of the literacy ability of participants should be based on the characteristics of the 
texts and tasks they are successful with, using the features of this framework, as in Figure 2. 

Figure 2: Relation of reporting framework to literacy framework 



Features of texts 
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